Skip to content

Prompt optimizations #6664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 19, 2025
Merged

Prompt optimizations #6664

merged 1 commit into from
May 19, 2025

Conversation

akatsoulas
Copy link
Collaborator

No description provided.

@akatsoulas akatsoulas requested review from escattone and Copilot May 16, 2025 10:00
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR optimizes the prompt instructions used for content moderation and topic classification in Mozilla's "{product}" support forums. Key changes include:

  • Updating setup.cfg to ignore E501 in the prompt file.
  • Overhauling the prompt in kitsune/llm/questions/prompt.py to clarify and improve the instructions for spam detection and classification.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
setup.cfg Adds per-file ignore for E501 specifically for the prompt file
kitsune/llm/questions/prompt.py Refines the prompt for both spam and topic classification with updated instructions
Comments suppressed due to low confidence (1)

kitsune/llm/questions/prompt.py:36

  • [nitpick] Consider defining a common constant for JSON formatting instructions instead of using the same placeholder in multiple locations to improve maintainability and reduce duplication.
{format_instructions}

Copy link
Contributor

@escattone escattone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r+wc

I really like your changes, especially the adjustments to the topic prompt. My only recommendation is to remove the formatting instruction preambles that you added, since they're not needed.



# Response format
Respond strictly with the following JSON structure:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The StructuredOutputParser.get_format_instructions() takes care of adding the following preamble (prior to the JSON schema):

The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "```json" and "```":

so this line can be removed.

- Look for keywords that align with topic descriptions and examples

# Response Format
Provide your classification using this exact JSON structure:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above.

- `100` = Completely certain.
4. Provide a concise explanation supporting your decision.


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps also remove this extra empty line?


```json
{topics}
```

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps keep this blank line to add some separation from the next section?

@akatsoulas akatsoulas merged commit b62e559 into mozilla:main May 19, 2025
1 of 2 checks passed
@akatsoulas akatsoulas deleted the prompt-updates branch May 19, 2025 07:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants